Search for: All records

We investigate pre-training techniques for abstractive multi-document summarization (MDS), which is much less studied than summarizing single documents. Though recent work has demonstrated the effectiveness of highlighting information salience for pretraining strategy design, they struggle to generate abstractive and reflective summaries, which are critical properties for MDS. To this end, we present PELMS, a pre-trained model that uses pre-training objectives based on semantic coherence heuristics and faithfulness constraints together with unlabeled multi-document inputs, to promote the generation of concise, fluent, and faithful summaries. To support the training of PELMS, we compile MultiPT, a multidocument pre-training corpus containing over 93 million documents to form more than 3 million unlabeled topic-centric document clusters, covering diverse genres such as product reviews, news, and general knowledge. We perform extensive evaluation of PELMS in lowshot settings on a wide range of MDS datasets. Our approach consistently outperforms competitive comparisons with respect to overall informativeness, abstractiveness, coherence, and faithfulness, and with minimal fine-tuning can match performance of language models at a much larger scale (e.g., GPT-4). 

Aspect-based sentiment analysis (ABSA) provides an opportunity to systematically generate user's opinions of specific aspects to enrich the idea creation process in the early stage of product/service design process. Yet, the current ABSA task has two major limitations. First, existing research mostly focusing on the subsets of ABSA task, e.g. aspect-sentiment extraction, extract aspect, opinion, and sentiment in a unified model is still an open problem. Second, the implicit opinion and sentiment are ignored in the current ABSA task. This article tackles these gaps by (1) creating a new annotated dataset comprised of five types of labels, including aspect, category, opinion, sentiment, and implicit indicator (ACOSI) and (2) developing a unified model which could extract all five types of labels simultaneously in a generative manner. Numerical experiments conducted on the manually labeled dataset originally scraped from three major e-Commerce retail stores for apparel and footwear products indicate the performance, scalability, and potentials of the framework developed. Several directions are provided for future exploration in the area of automated aspect-based sentiment analysis for user-centered design.